Decentralized Discrete-Time Neural Network Controller for a Class of Nonlinear Systems with Unknown Interconnections

A novel decentralized neural network (NN) controller in discrete-time is designed for a class of uncertain nonlinear discrete-time systems with unknown interconnections. Neural networks are used to approximate both the uncertain dynamics of the nonlinear systems and the unknown interconnections. Only local signals are needed for the decentralized controller design and the stability of the overall system can be guaranteed using the Lyapunov analysis. Further, controller redesign for the original subsystems is not required when additional subsystems are appended. Simulation results demonstrate the effectiveness of the proposed controller. The NN does not require an offline learning phase and the weights can be initialized at zero or randomly. Simulation results verify the theoretical conclusions

Reinforcement Learning Neural-Network-Based Controller for Nonlinear Discrete-Time Systems with Input Constraints

A novel adaptive-critic-based neural network (NN) controller in discrete time is designed to deliver a desired tracking performance for a class of nonlinear systems in the presence of actuator constraints. The constraints of the actuator are treated in the controller design as the saturation nonlinearity. The adaptive critic NN controller architecture based on state feedback includes two NNs: the critic NN is used to approximate the strategic utility function, whereas the action NN is employed to minimize both the strategic utility function and the unknown nonlinear dynamic estimation errors. The critic and action NN weight updates are derived by minimizing certain quadratic performance indexes. Using the Lyapunov approach and with novel weight updates, the uniformly ultimate boundedness of the closed-loop tracking error and weight estimates is shown in the presence of NN approximation errors and bounded unknown disturbances. The proposed NN controller works in the presence of multiple nonlinearities, unlike other schemes that normally approximate one nonlinearity. Moreover, the adaptive critic NN controller does not require an explicit offline training phase, and the NN weights can be initialized at zero or random. Simulation results justify the theoretical analysi

Reinforcement Learning-Based Output Feedback Control of Nonlinear Systems with Input Constraints

A novel neural network (NN) -based output feedback controller with magnitude constraints is designed to deliver a desired tracking performance for a class of multi-input-multi-output (MIMO) discrete-time strict feedback nonlinear systems. Reinforcement learning in discrete time is proposed for the output feedback controller, which uses three NN: 1) a NN observer to estimate the system states with the input-output data; 2) a critic NN to approximate certain strategic utility function; and 3) an action NN to minimize both the strategic utility function and the unknown dynamics estimation errors. The magnitude constraints are manifested as saturation nonlinearities in the output feedback controller design. Using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the state estimation errors, the tracking errors and weight estimates is shown

Adaptive Neural Network Control and Wireless Sensor Network Based Localization for UAV Formation

We consider a team of unmanned aerial vehicles (UAV\u27s) equipped with sensors and motes for wireless communication for the task of navigating to a desired location in a formation. First a neural network (NN)-based control scheme is presented that allows the UAVs to track a desired position and orientation with reference to the neighboring UAVs or obstacles in the environment. Second, we discuss a graph theory-based scheme for discovery, localization and cooperative control. The purpose of the NN cooperative controller is to achieve and maintain the desired formation shape in the presence of unmodeled dynamics and bounded unknown disturbances. Numerical results are included to illustrate the theoretical conclusion

Online Optimal Control of Affine Nonlinear Discrete-time Systems with Unknown Internal Dynamics by using Time-Based Policy Update

In this paper, the Hamilton-Jacobi-Bellman equation is solved forward-in-time for the optimal control of a class of general affine nonlinear discrete-time systems without using value and policy iterations. the proposed approach, referred to as adaptive dynamic programming, uses two neural networks (NNs), to solve the infinite horizon optimal regulation control of affine nonlinear discrete-time systems in the presence of unknown internal dynamics and a known control coefficient matrix. One NN approximates the cost function and is referred to as the critic NN, while the second NN generates the control input and is referred to as the action NN. the cost function and policy are updated once at the sampling instant and thus the proposed approach can be referred to as time-Based ADP. Novel update laws for tuning the unknown weights of the NNs online are derived. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded and that the approximated control signal approaches the optimal control input with small, bounded error over time. in the absence of disturbances, an optimal control is demonstrated. Simulation results are included to show the effectiveness of the approach. the end result is the systematic design of an optimal controller with guaranteed convergence that is suitable for hardware implementation. © 2012 IEEE

Near Optimal Neural Network-Based Output Feedback Control of Affine Nonlinear Discrete-Time Systems

In this paper, a novel online reinforcement learning neural network (NN)-based optimal output feedback controller, referred to as adaptive critic controller, is proposed for affine nonlinear discrete-time systems, to deliver a desired tracking performance. The adaptive critic design consist of three entities, an observer to estimate the system states, an action network that produces optimal control input and a critic that evaluates the performance of the action network. The critic is termed adaptive as it adapts itself to output the optimal cost-to-go function which is based on the standard Bellman equation. By using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the estimation and tracking errors and weight estimates is demonstrated. The effectiveness of the controller is evaluated for the task of nanomanipulation in a simulation environment

Neuro Emission Controller for Minimizing Cyclic Dispersion in Spark Ignition Engines

A novel neural network (NN) controller is developed to control spark ignition (SI) engines at extreme lean conditions. The purpose of neurocontroller is to reduce the cyclic dispersion at lean operation even when the engine dynamics are unknown. The stability analysis of the closed-loop control system is given and the boundedness of all signals is ensured. Results demonstrate that the cyclic dispersion is reduced significantly using the proposed controller. The neuro controller can also be extended to minimize engine emissions with high EGR levels, where similar complex cyclic dynamics are observed. Further, the proposed approach can be applied to control nonlinear systems that have similar structure as that of the engine dynamics

Spatial Diversity in Signal Strength Based WLAN Location Determination Systems

Literature indicates that spatial diversity can be utilized to compensate channel uncertainties such as multipath fading. Therefore, in this paper, spatial diversity is exploited for locating stationary and mobile objects in the indoor environment. First, space diversity technique is introduced for small scale motion and temporal variation compensation of received signal strength and it is demonstrated analytically that it enhances location accuracy. Small scale motion refers to movements of the transmitter and/or the receiver of the order of sub-wavelengths while temporal effects refer to environmental variations with time. A novel metric is introduced for selection combining in order to improve location accuracy through the addition of spatial diversity upon two available location determination schemes. The results are evaluated experimentally against single antenna system for reception by using low cost wireless RF devices such as motes. Alternatively, the impact of the number of location determination devices in a probabilistic WLAN network based on pre-profiling of signal strength is analyzed and it is demonstrated analytically that location accuracy improves with the number of receivers used. Spatial diversity in terms of the antenna spacing of 2lambda is evaluated and shown to provide a reduction in location determination error between 30 and 40% when compared to a single antenna system

Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation

In this paper, a novel reinforcement learning neural network (NN)-based controller, referred to adaptive critic controller, is proposed for affine nonlinear discrete-time systems with applications to nanomanipulation. In the online NN reinforcement learning method, one NN is designated as the critic NN, which approximates the long-term cost function by assuming that the states of the nonlinear systems is available for measurement. An action NN is employed to derive an optimal control signal to track a desired system trajectory while minimizing the cost function. Online updating weight tuning schemes for these two NNs are also derived. By using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the tracking error and weight estimates is shown. Nanomanipulation implies manipulating objects with nanometer size. It takes several hours to perform a simple task in the nanoscale world. To accomplish the task automatically the proposed online learning control design is evaluated for the task of nanomanipulation and verified in the simulation environmen

Adaptive Critic Neural Network-Based Object Grasping Control using a Three-finger Gripper

Grasping of objects has been a challenging task for robots. The complex grasping task can be defined as object contact control and manipulation subtasks. In this paper, object contact control subtask is defined as the ability to follow a trajectory accurately by the fingers of a gripper. The object manipulation subtask is defined in terms of maintaining a predefined applied force by the fingers on the object. A sophisticated controller is necessary since the process of grasping an object without a priori knowledge of the object\u27s size, texture, softness, gripper, and contact dynamics is rather difficult. Moreover, the object has to be secured accurately and considerably fast without damaging it. Since the gripper, contact dynamics, and the object properties are not typically known beforehand, an adaptive critic neural network (NN)-based hybrid position/force control scheme is introduced. The feedforward action generating NN in the adaptive critic NN controller compensates the nonlinear gripper and contact dynamics. The learning of the action generating NN is performed on-line based on a critic NN output signal. The controller ensures that a three-finger gripper tracks a desired trajectory while applying desired forces on the object for manipulation. Novel NN weight tuning updates are derived for the action generating and critic NNs so that Lyapunov-based stability analysis can be shown. Simulation results demonstrate that the proposed scheme successfully allows fingers of a gripper to secure objects without the knowledge of the underlying gripper and contact dynamics of the object compared to conventional schemes